The basic concept is that a data filter is capable of taking a set of data, process this data and output something else.
For example, there could be selective filters which select a particular set of data from the original, like a range, a specific value like a maximum, minimum, etc.
There could also be filters that slightly modify the input data, giving out the effect of "smoothing out" the data, filtering "bad" values, etc.
The concept is open enough that any kind of filter can be written, and they can be combined to obtained the desired effect.
The examples we have so far are entirely numeric, so the filters take a set of data in pairs (x,y) and return a numeric set of data. But in theory, filters could return anything, including descriptive text.

A Data Filter is an entity that takes a DataStore (inputDS) and returns another DataStore (outputDS). The only requirement is that the resultant data store has to be "live", meaning that if the values in inputDS change, the same instance of the outputDS once returned by the filter has to change its values accordingly. Also, if the inputDS is changed, the outputDS should change too and it should not be necessary to ask for a new outputDS instance from the filter again.
This allows the possibility of chaining filters together without having to disconnect/connect them again when the inputDS changes.

The DataFilter interface

The main interfaces used for the data filter framework are currently in the package org.concord.smartgraph in the OTrunkDataUtil project. We should probably move it somewhere else.
The main methods of the DataFilter interface are:

 
public void setInputDataStore(DataStore ds);
public DataStore getResultDataStore();

which basically set the input data store and retrieve the output data store (very self-explanatory).
There is also an extra method that allows the configuration of a filter, passing any kind of parameters as needed.

public void setFilterDescription(DataFilterDescription desc);

A DataFilterDescription is another interface that allows the configuration of any data filter. The idea is that, on top of some basic properties that all filters should have, it can also store any additional property in the form of name, value.

There are default implementations for these two interfaces that make it a lot easier to write filters. Skipping all further explanation, here's an exameple of how to use some of the default filters that are already written.

Example No. 1: How to use filters

This code is taken from the class SimpleExamplePanel in the package org.concord.examples.smartgraph in the Examples project.
It uses a Maximum filter, which simply takes a set of data organized in x,y values and selects the point(s) that have the maximum y value.

//Create an initial data store to play around with
PointsDataStore originalData = new PointsDataStore();
originalData.addPoint(0, 0);
originalData.addPoint(1, 10);
originalData.addPoint(2, 5);

//Add the initial data store to a table so we can see the values
DataTablePanel table1 = new DataTablePanel();
table1.getTableModel().addDataStore(originalData);
		
//Create a "maximum" filter (capable of choosing the maximum y value in the set)
DataFilter filter = new DataMaximumFilter();
filter.setFilterDescription(new DefaultDataFilterDescription("maximum"));
		
//Plug in the input data store into the filter
filter.setInputDataStore(originalData);
		
//Get the output data store from the filter (which should contain the maximum)
DataStore filteredData = filter.getResultDataStore();
		
//Add the resultant data store to a second table so we can see it
DataTablePanel table2 = new DataTablePanel();
table2.getTableModel().addDataStore(filteredData);

Now you have two tables (JPanels) that you can display and see how it works. The first table shows the original data and the second table shows the filtered data (the maximum value). You can manually change the values on the first table and see how the second table changes. Notice that when more than one point has the same y value and it's the maximum, the second table will have more than one value.

You can run this example here.

How to create a new filter

Instead of implementing the basic interfaces (which you can also do if you prefer to), there is a more convenient way of writing filters, using the default implementations that are provided in the smartgraph package.

The in-place filter

One type of filter that may be easily understood is a filter that works like a "function". It takes a point (x,y) and it returns a new y value for that point.
For example, let's say there is a set of data with really small y values (less than 1, for example). You would like to make those values "bigger", for example, multiplying all the values by 1000. Then, your filter would just take a point (x,y) and return (x, y * 1000). The resultant effect is amplified data.

In order to implement filters that will take (x,y) points and return points of the form (x, <something that can be calculated with x and y only>), there is an easy way of doing it. Just extend the InPlaceDataFilter class which has only one abstract function (you are required to implement only one thing). Of course, you can override more mehtods if you need more flexibility or more functionality.

Example No. 2: Extending InPlaceDataFilter to write a simple filter

Let's write our little "1000x amplifier" example:

public class DataAmplifier extends InPlaceDataFilter
{
    public float getYValue(int numSample, float xValue, float inputYValue)
    {
        //We don't care about the sample number OR the x value, we just want to amplify the y value by 1000:
        return inputYValue * 1000;
    }
}

That's it! We can now pass a data store through our amplified filter and see those values shoot up!
Now, maybe 1000 is too much... actually, depending on the case, sometimes we would like to amplify the data more or less. In order to do that, we could modify the filter and add a parameter to it that specifies the "amplification factor".
We can use the filter description for that. We can just assume the user will specify the factor in the filter description, otherwise we will use our default.

public class DataAmplifier extends InPlaceDataFilter
{
	public final String PROPERTY_MAGNIFICATION = "magnification";
    public final int DEFAULT_MAGNIFICATION = 1000;

    public float getYValue(int numSample, float xValue, float inputYValue)
    {
        //We don't care about the sample number OR the x value, we just want to amplify the y value
        return inputYValue * getMagnificationFactor();
    }

    public float getMagnificationFactor()
    {
    	try{
	    	DataFilterDescription fDesc = getFilterDescription();
	    	String val = fDesc.getProperty(PROPERTY_MAGNIFICATION);
	    	return Float.parseFloat(val);
    	}
    	catch(Exception ex){
    		//If there is no filter description (null)
    		//OR if the value of the property cannot be parsed into a float, then:
    		return DEFAULT_MAGNIFICATION;
    	}
    }
}

The code looks a little more complicated, basically because we are handling the property carefully.
Now let's see how would somebody use this filter, specifically, how would somebody specify a new magnification property for it.

//Like example No.1, let's assume we have our initial data store in a variable called originalData

//Create the amplifier filter and set it up
DataFilter filter = new DataAmplifier();
//We use the DefaultDataFilterDescription for convenience, simply passing a label that identifies the filter.  
FilterDescription filterDesc = new DefaultDataFilterDescription("amplifier");
filterDesc.setProperty(DataAmplifier.PROPERTY_MAGNIFICATION, "10");
filter.setFilterDescription(filterDesc);

//Plug in the input data store into the filter
filter.setInputDataStore(originalData);
		
//Get the output data store from the filter (which should now contain the 10x amplified data!)
DataStore filteredData = filter.getResultDataStore();

The code above sets up a filter that amplifies the data (y values) by a factor of 10.